Simple and interpretable discrimination

Riassunto: Several authors proposed a number of approaches for constructing easily interpretable components still preserving the nice features of the principal components. The work employs one such approach to produce interpretable canonical variates and explore their discrimination and classification behavior. Discriminant analysis (DA) is a descriptive multivariate technique for analyzing grouped data, i.e. the rows of the data matrix are divided into a number of groups that usually represent samples from different populations (Fisher, 1936). Recently DA has also been viewed as a promising dimensionality reduction technique. Indeed, the presence of group structure in the data additionally facilitates dimensionality reduction. The best known variety of DA is linear discriminant analysis (LDA), whose central goal is to describe the differences between the groups in terms of discriminant functions defined as linear combinations of the original variables (Fisher, 1936). The interpretation of the discriminant functions is based on the coefficients of the original variables in the linear combinations. The problem is similar to interpretation of principal components (Trendafilov and Jolliffe, 2006): the interpretation can be clear if there are only few large coefficients and the rest are all close to or exactly zero. Unfortunately, in many applications this is not the case. There are several approaches to the interpretation of the discriminant functions, each of which has disadvantages (Rencher, 1992; Trendafilov and Jolliffe, 2006a). A modification of LDA aiming better discrimination and possibly interpretation is considered in (Krzanowski, 1995): the canonical variates are constrained to be orthogonal in the original data space. Recently, there is an increasing interest in approximate but computationally simple and easy to interpret versions of principal component analysis Roughly speaking, the aim of all these works is to find " components " with considerable number of exact zero coefficients. The reason is twofold: one may need such simplified PCA for either interpretation or coping with very large data. In this work we extend this strategy to LDA, where the interpretation problem seems even more serious that in PCA: indeed, the interpretation relies on three types of coefficients (raw, standardized and structure) which is quite often controversial (Rencher, 1992; Trendafilov and Jolliffe, 2006a). One such approach for interpretation of the discriminant functions is already proposed by Trendafilov and Jolliffe (2006a). The classical LDA problem is subject to additional LASSO constraints requiring that the sum of the absolute values of the coefficients be less than some pre-specified threshold t (Tibshirani, …